Classification of mental workload using brain connectivity and machine learning on electroencephalogram data.

Mental workload refers to the cognitive effort required to perform tasks, and it is an important factor in various fields, including system design, clinical medicine, and industrial applications. In this paper, we propose innovative methods to assess mental workload from EEG data that use effective brain connectivity for the purpose of extracting features, a hierarchical feature selection algorithm to select the most significant features, and finally machine learning models. We have used the Simultaneous Task EEG Workload (STEW) dataset, an open-access collection of raw EEG data from 48 subjects. We extracted brain-effective connectivities by the direct directed transfer function and then selected the top 30 connectivities for each standard frequency band. Then we applied three feature selection algorithms (forward feature selection, Relief-F, and minimum-redundancy-maximum-relevance) on the top 150 features from all frequencies. Finally, we applied sevenfold cross-validation on four machine learning models (support vector machine (SVM), linear discriminant analysis, random forest, and decision tree). The results revealed that SVM as the machine learning model and forward feature selection as the feature selection method work better than others and could classify the mental workload levels with accuracy equal to 89.53% (± 1.36).

Emotion recognition in EEG signals using deep learning methods: A review.

Emotions are a critical aspect of daily life and serve a crucial role in human decision-making, planning, reasoning, and other mental states. As a result, they are considered a significant factor in human interactions. Human emotions can be identified through various sources, such as facial expressions, speech, behavior (gesture/position), or physiological signals. The use of physiological signals can enhance the objectivity and reliability of emotion detection. Compared with peripheral physiological signals, electroencephalogram (EEG) recordings are directly generated by the central nervous system and are closely related to human emotions. EEG signals have the great spatial resolution that facilitates the evaluation of brain functions, making them a popular modality in emotion recognition studies. Emotion recognition using EEG signals presents several challenges, including signal variability due to electrode positioning, individual differences in signal morphology, and lack of a universal standard for EEG signal processing. Moreover, identifying the appropriate features for emotion recognition from EEG data requires further research. Finally, there is a need to develop more robust artificial intelligence (AI) including conventional machine learning (ML) and deep learning (DL) methods to handle the complex and diverse EEG signals associated with emotional states. This paper examines the application of DL techniques in emotion recognition from EEG signals and provides a detailed discussion of relevant articles. The paper explores the significant challenges in emotion recognition using EEG signals, highlights the potential of DL techniques in addressing these challenges, and suggests the scope for future research in emotion recognition using DL techniques. The paper concludes with a summary of its findings.

A Hybrid EEG-based Emotion Recognition Approach Using Wavelet Convolutional Neural Networks and Support Vector Machine.

Introduction: Nowadays, deep learning and convolutional neural networks (CNNs) have become widespread tools in many biomedical engineering studies. CNN is an end-to-end tool, which makes the processing procedure integrated, but in some situations, this processing tool requires to be fused with machine learning methods to be more accurate. Methods: In this paper, a hybrid approach based on deep features extracted from wavelet CNNs (WCNNs) weighted layers and multiclass support vector machine (MSVM) was proposed to improve the recognition of emotional states from electroencephalogram (EEG) signals. First, EEG signals were preprocessed and converted to Time-Frequency (T-F) color representation or scalogram using the continuous wavelet transform (CWT) method. Then, scalograms were fed into four popular pre-trained CNNs, AlexNet, ResNet-18, VGG-19, and Inception-v3 to fine-tune them. Then, the best feature layer from each one was used as input to the MSVM method to classify four quarters of the valence-arousal model. Finally, the subject-independent leave-one-subject-out criterion was used to evaluate the proposed method on DEAP and MAHNOB-HCI databases. Results: Results showed that extracting deep features from the earlier convolutional layer of ResNet-18 (Res2a) and classifying using the MSVM increased the average accuracy, precision, and recall by about 20% and 12% for MAHNOB-HCI and DEAP databases, respectively. Also, combining scalograms from four regions of pre-frontal, frontal, parietal, and parietal-occipital and two regions of frontal and parietal achieved the higher average accuracy of 77.47% and 87.45% for MAHNOB-HCI and DEAP databases, respectively. Conclusion: Combining CNN and MSVM increased the recognition of emotion from EEG signals and the results were comparable to state-of-the art studies.

Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals.

Convolutional Neural Networks (CNN) have recently made considerable advances in the field of biomedical signal processing. These methodologies can assist in emotion recognition for affective brain computer interface. In this paper, a novel emotion recognition system based on the effective connectivity and the fine-tuned CNNs from multichannel Electroencephalogram (EEG) signal is presented. After preprocessing EEG signals, the relationships among 32 channels of EEG in the form of effective brain connectivity analysis which represents information flow between regions are computed by direct Directed Transfer Function (dDTF) method which yields a 32*32 image. Then, these constructed images from EEG signals for each subject were fed as input to four versions of pre-trained CNN models, AlexNet, ResNet-50, Inception-v3 and VGG-19 and the parameters of these models are fine-tuned, independently. The proposed deep learning architectures automatically learn patterns in the constructed image of the EEG signals in frequency bands. The efficiency of the proposed approach is evaluated on MAHNOB-HCI and DEAP databases. The experiments for classifying five emotional states show that the ResNet-50 applied on dDTF images in alpha band achieves best results due to specific architecture which captures the brain connectivity, efficiently. The accuracy and F1-score values for MAHNOB-HCI were obtained 99.41, 99.42 and for DEAP databases, 98.17, and 98.23. Newly proposed model is capable of effectively analyzing the brain function using information flow from multichannel EEG signals using effective connectivity measure of dDTF and ResNet-50.

Detection of schizophrenia using hybrid of deep learning and brain effective connectivity image from electroencephalogram signal.

Detection of mental disorders such as schizophrenia (SZ) through investigating brain activities recorded via Electroencephalogram (EEG) signals is a promising field in neuroscience. This study presents a hybrid brain effective connectivity and deep learning framework for SZ detection on multichannel EEG signals. First, the effective connectivity matrix is measured based on the Transfer Entropy (TE) method that estimates directed causalities in terms of brain information flow from 19 EEG channels for each subject. Then, TE effective connectivity elements were represented by colors and formed a 19 × 19 connectivity image which, simultaneously, represents the time and spatial information of EEG signals. Created images are used to be fed into the five pre-trained Convolutional Neural Networks (CNN) models named VGG-16, ResNet50V2, InceptionV3, EfficientNetB0, and DenseNet121 as Transfer Learning (TL) models. Finally, deep features from these TL models equipped with the Long Short-Term Memory (LSTM) model for the extraction of most discriminative spatiotemporal features are used to classify 14 SZ patients from 14 healthy controls. Results show that the hybrid framework of pre-trained CNN-LSTM models achieved higher accuracy than pre-trained CNN models. The highest average accuracy and F1-score were achieved using the EfficientNetB0-LSTM model through the 10-fold cross-validation method equal to 99.90% and 99.93%, respectively. Therefore, the superior performance of the hybrid framework of brain effective connectivity images from EEG signals and pre-trained CNN-LSTM models show that the proposed method is highly capable of detecting SZ patients from healthy controls.

Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals.

Schizophrenia (SZ) is a severe disorder of the human brain which disturbs behavioral characteristics such as interruption in thinking, memory, perception, speech and other living activities. If the patient suffering from SZ is not diagnosed and treated in the early stages, damage to human behavioral abilities in its later stages could become more severe. Therefore, early discovery of SZ may help to cure or limit the effects. Electroencephalogram (EEG) is prominently used to study brain diseases such as SZ due to having high temporal resolution information, and being a noninvasive and inexpensive method. This paper introduces an automatic methodology based on transfer learning with deep convolutional neural networks (CNNs) for the diagnosis of SZ patients from healthy controls. First, EEG signals are converted into images by applying a time-frequency approach called continuous wavelet transform (CWT) method. Then, the images of EEG signals are applied to the four popular pre-trained CNNs: AlexNet, ResNet-18, VGG-19 and Inception-v3. The output of convolutional and pooling layers of these models are used as deep features and are fed into the support vector machine (SVM) classifier. We have tuned the parameters of SVM to classify SZ patients and healthy subjects. The efficiency of the proposed method is evaluated on EEG signals from 14 healthy subjects and 14 SZ patients. The experiments showed that the combination of frontal, central, parietal, and occipital regions applied to the ResNet-18-SVM achieved best results with accuracy, sensitivity and specificity of 98.60% ± 2.29, 99.65% ± 2.35 and 96.92% ± 2.25, respectively. Therefore, the proposed method as a diagnostic tool can help clinicians in detection of the SZ patients for early diagnosis and treatment.